1 00:00:00,160 --> 00:00:01,050 Ha ha right. 2 00:00:01,080 --> 00:00:09,420 So in this lesson we're finally gonna start talking about tenses the tenser and tenser flow. 3 00:00:09,650 --> 00:00:11,370 Now what is the tensor. 4 00:00:11,370 --> 00:00:18,240 I find that googling for the definition or for other people's explanation often comes back with quite 5 00:00:18,480 --> 00:00:21,570 difficult to understand definitions and results. 6 00:00:21,810 --> 00:00:28,890 And the reason is that yeah tenses are somewhat complicated and abstract but they're also used in many 7 00:00:28,890 --> 00:00:35,280 different fields outside of neural networks and machine learning and the thing is you tend to get very 8 00:00:35,280 --> 00:00:38,430 different explanations depending on who you're asking. 9 00:00:38,430 --> 00:00:40,520 Everybody is gonna give you their point of view. 10 00:00:40,710 --> 00:00:45,720 So a mathematician is going to have a very different starting point for explaining what a tensor is 11 00:00:46,060 --> 00:00:51,930 than say a software engineer and a physicist will also give you a very very different examples for the 12 00:00:51,930 --> 00:00:59,160 uses of tenses than say a civil engineer from our perspective we're going to be working with data and 13 00:00:59,160 --> 00:01:06,810 numbers right so that we can start to understand what a tensor is is that it's essentially a way of 14 00:01:06,810 --> 00:01:10,260 storing data in some kind of structure. 15 00:01:10,440 --> 00:01:16,080 If we only had a single number then things would be easy on the other hand if we had lots of numbers 16 00:01:16,590 --> 00:01:22,710 then we have to make a choice as to how to represent those numbers one way that we've seen already is 17 00:01:22,710 --> 00:01:26,220 a list or a single column of numbers. 18 00:01:26,220 --> 00:01:31,600 We sometimes referred to that as a vector a computer scientists on the other hand would look at the 19 00:01:31,600 --> 00:01:37,590 same thing and call it an array and this array only has one dimension. 20 00:01:37,960 --> 00:01:44,560 Sometimes on the other hand we had more numbers and we were working with data frames or matrices. 21 00:01:44,560 --> 00:01:47,890 In this case our data was structured like in an Excel spreadsheet. 22 00:01:48,430 --> 00:01:49,790 So we had two dimensions. 23 00:01:49,930 --> 00:01:54,010 Rows and columns and then things got real fancy. 24 00:01:54,070 --> 00:01:56,290 When we added that third dimension. 25 00:01:56,500 --> 00:02:01,110 Remember how with color images we had the red green and blue channel. 26 00:02:01,120 --> 00:02:05,690 In that case we started stacking these spreadsheets on top of one another. 27 00:02:05,830 --> 00:02:08,750 And that's because we had three dimensions. 28 00:02:09,160 --> 00:02:14,080 When we're talking about tenses and we're talking about these dimensions the number of dimensions actually 29 00:02:14,080 --> 00:02:15,000 has a name. 30 00:02:15,040 --> 00:02:17,560 It's called the rank. 31 00:02:17,560 --> 00:02:28,030 So a single number or a scalar has a rank of 0 a vector or a column of numbers would have a rank of 32 00:02:28,030 --> 00:02:29,040 1. 33 00:02:29,200 --> 00:02:36,570 The Matrix would have a rank of 2 and our stacked matrix would have a rank of 3. 34 00:02:36,820 --> 00:02:42,520 Now when working with tensor flow we're actually working with a data structure that can have any number 35 00:02:42,520 --> 00:02:43,720 of dimensions. 36 00:02:43,900 --> 00:02:50,100 So a tensor is basically a container with and dimensions. 37 00:02:50,170 --> 00:02:55,290 So now we have a proper name for that stacked spreadsheet of our GDP values. 38 00:02:55,360 --> 00:03:02,350 We are essentially working with a three dimensional tensor so the way to think about a tensor from our 39 00:03:02,350 --> 00:03:10,960 perspective is a generalization of our friendly neighborhood matrix to end dimensions from two dimensions 40 00:03:11,140 --> 00:03:17,950 to end dimensions and because we already have words for one dimensional and two dimensional tenses like 41 00:03:18,220 --> 00:03:19,680 vector and Matrix. 42 00:03:19,750 --> 00:03:24,400 You're typically only see the word tensor used when people talk about anything with three dimensions 43 00:03:24,460 --> 00:03:31,720 or more but the thing is there's actually more to tenses than just the way of holding data tensor is 44 00:03:31,810 --> 00:03:39,070 also follow certain mathematical rules meaning you can multiply and add tenses together and generally 45 00:03:39,070 --> 00:03:40,300 do linear algebra. 46 00:03:40,300 --> 00:03:46,090 Similar to how we were multiplying matrices together and this kind of linear algebra is gonna come in 47 00:03:46,090 --> 00:03:52,670 really handy when we are calculating the input and the output values of our neurons. 48 00:03:52,780 --> 00:03:59,620 An example of the kind of calculation we will do is multiplying the output values of one layer of neurons 49 00:04:00,040 --> 00:04:04,960 times the connection weights to figure out what the input values should be for the next layer. 50 00:04:05,680 --> 00:04:06,980 So let's walk through this. 51 00:04:07,000 --> 00:04:12,220 Suppose we've got these two layers right here and that blue layer that first layer is going to have 52 00:04:12,220 --> 00:04:13,870 a certain number of outputs. 53 00:04:14,140 --> 00:04:16,840 So that first neuron might have an output of two point two. 54 00:04:16,860 --> 00:04:19,130 The second one seven point one. 55 00:04:19,360 --> 00:04:21,100 And so on. 56 00:04:21,100 --> 00:04:24,180 Now I'm going to substitute some symbols instead of these numbers. 57 00:04:24,310 --> 00:04:30,710 I'm just gonna call these outputs output one output to up with three up until output 5 and similarly 58 00:04:30,770 --> 00:04:33,440 I'm going to give the inputs in that second layer and name two. 59 00:04:33,470 --> 00:04:40,980 I'm going to call these ones input 1 into in 3 and 4 so here's the question how would we calculate the 60 00:04:40,980 --> 00:04:47,840 values of these inputs for the green layer remember these inputs are the same ones that are going to 61 00:04:47,840 --> 00:04:54,050 go into the activation functions of that green neuron to determine their output in turn. 62 00:04:54,050 --> 00:04:55,120 But that's the next step. 63 00:04:56,210 --> 00:04:57,990 So let's walk through this. 64 00:04:58,190 --> 00:05:02,550 If we only had to worry about the connection weights we're gonna ignore biases for now. 65 00:05:02,660 --> 00:05:08,510 Then we would use some linear algebra we would take all of those weights and then we would multiply 66 00:05:08,510 --> 00:05:12,340 them by the outputs from that first layer. 67 00:05:12,770 --> 00:05:16,550 And that's what we would get the inputs to the second layer. 68 00:05:16,560 --> 00:05:20,090 The key here is to multiply the output of the first layer. 69 00:05:20,130 --> 00:05:26,930 Times the weights if you look at these weights here you can see I've put low subscript next to them. 70 00:05:27,010 --> 00:05:36,160 So the interesting thing is here that that first row weight 1 1 weight one to weight 1 3 basically correspond 71 00:05:36,160 --> 00:05:39,600 to the connection weights of this neuron and there's five of them. 72 00:05:39,600 --> 00:05:39,940 Right. 73 00:05:39,940 --> 00:05:41,790 One two three four five. 74 00:05:41,840 --> 00:05:48,100 Because this neuron is receiving an input from these five blue neurons. 75 00:05:48,100 --> 00:05:50,920 So weight 1 1 will be multiplied with output one. 76 00:05:50,920 --> 00:05:57,340 Weight 1 2 will be multiplied without put 2 and all of these then will be added up and then we get that 77 00:05:57,340 --> 00:06:04,570 sum which is gonna be the weighted input for this neuron and the same thing will happen to that second 78 00:06:04,570 --> 00:06:11,200 row that second row of weights will correspond to all of these weights right here for neuron number 79 00:06:11,230 --> 00:06:18,560 two road number three will correspond to these weights here for neuron number three and this one for 80 00:06:18,560 --> 00:06:24,860 near a number for the columns on the other hand correspond to these weights right here all connected 81 00:06:25,010 --> 00:06:31,760 to that first blue and you're on the second column will be these for the third column will be these 82 00:06:31,760 --> 00:06:38,980 for weights in essence this is the calculation that takes place when we talk about the neurons taking 83 00:06:39,040 --> 00:06:46,030 a weighted average of their inputs let's keep this calculation in mind as we head back to our Jupiter 84 00:06:46,030 --> 00:06:49,390 notebook and set up our tensor is and our layers.